# Replication Files for "When Conservatives See Red but Liberals Feel Blue: Labeler Characteristics and Variation in Content Annotation."


## Set-up
Run with R version 4.4.2 and package versions (in order of appearance in scripts, all newly installed in December 2024):

tidyverse   2.0.0

xtable  1.8-4

irr   0.84.1

betaDelta   1.0.5

stargazer   5.2.3

dplyr   2.5.0

tidyr   1.3.1

rio   1.2.3

stringr   1.5.1

irr   0.84.1

ggplot2   3.5.1

broom   1.0.7 

lme4  1.1-35.5

hrbrthemes  0.8.7

ggsci   3.2.0

## Description of folders and files

1. Code folder: Contains scripts to replicate paper findings, including appendices:
* 1_meta_analysis: Replicates in-text findings on how often the analyzed papers reported IRR and labeler demographics, plus tables in Appendix A
* 2_image_annotation_diffs: Replicates Figures 1(A), 2, and 3
* 3_image_regressions: Replicates Figure 1(B), 4, plus all results in Appendices D and E (Appendix Figures 2-8, Tables 6-7)
* 4_text_study_figure_5: Replicates Figure 5
* 5_text_study_figure_6: Replicates Figure 6
* 6_text_study_figure_9_appF: Replicates Appendix F, Figure 9
* 7_text_study_figure_10_appG: Replicates Appendix G, Figure 10
* 8_text_study_figure_11_appH: Replicates Appendix H, Figure 11
* 9_meta_analysis_figure_1_appA: Replicates Appendix A, Figure 1

2. Data folder: Contains all replication data files

| Datasets included for replication:              | Empirical portion of paper: | Used in script(s):                                                                   |
| ----------------------------------------------- | --------------------------- | ------------------------------------------------------------------------------------ |
| meta_analysis_master.csv                        | Meta analysis               | 1_meta_analysis.R                                                                    |
| clean_meta_data.csv                             | Meta analysis               | 1_meta_analysis.R                                                                    |
| clean_meta_data_w_std_irr_info.csv              | Meta analysis               | 9_meta_analysis_figure_8_appA.R                                                      |
| slim_image_labels.csv                           | Image example               | 2_image_annotation_diffs.R                                                           |
| all_tweets_w_labeled_images.csv                 | Image example               | 3_image_regressions.R                                                                |
| text-experiment-merged-dataset-PAIRWISE-IRR.csv | Text example                | 4_text_study_figure_5.R; 6_text_study_figure_9_appF.R; 7_text_study_figure_10_appG.R |
| text-experiment-merged-dataset.csv              | Text example                | 4_text_study_figure_5.R; 5_text_study_figure_6.R; 8_text_study_figure_11_appH.R      |
| SESSION_01_data_GOLD.csv                        | Text example                | 8_text_study_figure_11_appH.R                                                        |